In recent years, along with development of an information communication technology, an image processing technology based on an augmented reality (AR) for adding visual display information to an image obtained by shooting an actual space (external world) by using a computer to display the image has been developed. An interface in the technology related to the augmented reality is provided with the following function. That is, shooting of a marker (which may also be referred to as a two-dimensional code) provided to an object in front of user's eyes is performed by using a camera fixed at an arbitrary position or a freely movable camera to obtain an image including the marker. Thereafter, display information corresponding to additional information related to the object and also being associated with the marker is displayed on a display.
An image processing method based on the augmented reality is characterized, specifically, in that the display information corresponding to the marker is superposed and displayed on a display of a camera (which may also be referred to as a preview screen), a display of a PC to which a camera is mounted, or a display of a mobile terminal based on a reference position determined from four corners of the shot marker and a physical coordinate system based on the reference position. According to the image processing method, it is possible to associate the display information with a display position of the object.
Functions of identifying a failure location when a failure occurs in an electronic device or the like and supporting field work such as a user failure recovery operation are realized by using the augmented reality technology. For example, a marker having a particular shape feature is provided to an object set as a work target or an area in the vicinity of this object, and the display information is superposed in real time on the image shot by the camera to be displayed based on various pieces of information included in the marker. For example, in supporting of a recovery operation for a paper jam failure in a copier, a technology has been proposed for performing superposed display of an internal video of the copier associated with a paper jam occurrence position and an operation procedure on the copier corresponding to the work target.
Herein, it is supposed that the following various problems may exist in the field operation based on the related-art method in which the image processing technology of the augmented reality in the related art is not used.
(1-1) An operator takes in to a work site a large number of written operation procedures made of paper in which work target positions and procedures are described. The operator selects, by itself, a part to be used and referred to and also executes the operation while visually comparing the descriptions of the written operation procedures with a site situation. For this reason, it takes time to select the part to be used, and an operational error occurs in a manner that an erroneous part is selected or an erroneous visual recognition occurs upon the visual comparison, for example.
(1-2) A work results recorded by filling in a check sheet made of paper. Since work load for digitalizing recording contents is large, the work result is saved as it is in the form of the check sheet made of paper. Thus, it is difficult to perform organization and analysis of the work result.
With regard to the above-described problems, the following improvement effects are expected by applying the image processing technology using the augmented reality to the above-described work site.
(2-1) Only by shooting a work target object by using the camera of the image processing apparatus that stores digitalized manuals, operation parts and procedures related to the work target object are displayed and presented in real time by way of superposed display of display information (which may also be referred to as an object icon) on an image of the display (which may also be referred to as a camera preview image). Accordingly, it is possible to shorten working hours of the user and reduce operational errors.
(2-2) A recording operation of the work result is executed by pressing the display information displayed while being associated with the work target object to be selected by a finger, for example, to switch the screen to an input screen for the work result. When the recording of the work result is completed, since the work result has been already digitalized, it is facilitated to perform the organization and editing of data of the work result. In this manner, according to the image processing technology using the augmented reality, various improvements in working efficiencies in the work site are expected.
For example, “Proposal and Evaluation of Decommissioning Support Method of Nuclear Power Plants using Augmented Reality”, Collected Papers of The Virtual Reality Society of Japan, Vol. 13, No. 2, pp. 289-300, 2008 or the like is disclosed as a related art.
With regard to a display operation of the display information in the image processing technology using the augmented reality based on a current marker recognition, for example, when recognition of a marker included in the image is successful, a recognition position and an orientation of the marker included in the image are used as a reference, and the display information is displayed based on a relative position with respect to the reference position of the marker. As described above, the reference position of the marker is determined, for example, from four corners of the marker. Since the display position of the display information is specified in accordance with the reference position of the marker, it is important to improve a detection accuracy of the reference position of the marker in order to secure visibility to the user of the display information. From the above descriptions, the visibility to the user of the display information may be preferably further improved.